OPTIMAL  GUIDANCE  OF  A  RELAY  MAV  FOR  ISR 
SUPPORT  BEYOND  LINE-OF-SIGHT 

THESIS 

John  H.  Hansen,  Second  Lieutenant,  USAF 
AFIT/GAE/ENG/08-0 1 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  United 
States  Government. 


AFIT/GAE/ENG/08-01 


OPTIMAL  GUIDANCE  OF  A  RELAY  MAY  FOR  ISR 
SUPPORT  BEYOND  LINE-OF-SIGHT 


THESIS 


Presented  to  the  Faculty 

Department  of  Electrical  and  Computer  Engineering 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Aeronautical  Engineering 

John  H.  Hansen,  BS 
Second  Lieutenant,  USAF 

March  2008 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT/GAE/ENG/08-01 


OPTIMAL  GUIDANCE  OF  A  RELAY  MAV  FOR  ISR 
SUPPORT  BEYOND  LINE-OF-SIGHT 


John  H.  Hansen,  BS 
Second  Lieutenant,  USAF 


Approved: 


Dr.  Meir  Pachter  (Chairman) 


/Z/2coZ 

Date 


Maj.  Paul  Blue  (Member) 


Date 


Date 


AFIT/GAE/ENG/08-01 


Abstract 

This  thesis  developed  guidance  laws  to  optimally  position  a  relay  Micro-UAV 
(MAV)  to  provide  an  operator  with  real-time  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  by  relaying  communication  and  video  signals  when  there  is  no 
line-of-sight  between  the  operator  at  the  base  and  the  rover  MAV  performing  the  ISR 
mission. 

The  ISR  system  consists  of  two  MAVs,  the  Relay  and  the  Rover,  and  a  Base.  The 
Relay  strives  to  position  itself  to  minimize  the  radio  frequency  (RF)  power  required  for 
maintaining  communications  between  the  Rover  and  the  Base,  while  the  Rover  performs 
the  ISR  mission,  which  may  maximize  the  required  RF  power.  The  optimal  control  of  the 
Relay  MAV  then  entails  the  solution  of  a  differential  game.  Applying  Pontryagin’s 
Maximum  Principle  yields  a  nonlinear  Two-Point  Boundary  Value  Problem  (TPBVP). 

Suboptimal  solutions  are  also  analyzed  to  aid  in  solving  the  TPBVP  which  yields 
the  solution  of  the  differential  game.  One  suboptimal  approach  is  based  upon  the 
geometry  of  the  ISR  system.  Another  suboptimal  approach  envisions  a  stationary  Rover 
and  solves  for  the  optimal  path  for  the  Relay.  Both  suboptimal  approaches  showed  that 
the  optimal  path  for  the  Relay  is  to  head  straight  toward  the  midpoint  between  the  Rover 
and  the  Base. 
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OPTIMAL  GUIDANCE  OF  A  RELAY  MAY  FOR  ISR 


SUPPORT  BEYOND  LINE-OF-SIGHT 

1  Introduction 

1.1  Operational  Background 

Lessons  learned  from  OPERATION  ENDURING  FREEDOM  and  OPERATION 
IRAQI  FREEDOM  have  demonstrated  a  need  for  teams  on  the  ground  in  urban 
environments  to  organically  engage  high  value,  time-sensitive  targets  in  real-time,  from 
Near  Line-of-Sight  (NLOS)  ranges  (500  m  to  5  km)  without  waiting  for  outside  air 
support.  Currently,  engaging  these  NLOS  targets  requires  coordination  of  orbiting  assets 
such  as  fighter  or  bomber  aircraft,  Hellfire  missile  equipped  Predator  Unmanned  Aerial 
Vehicles  (UAVs),  or  joint  ground-based  artillery  systems.  While  devastatingly  effective, 
these  systems  have  three  drawbacks:  (1)  they  must  be  on-station  and  available  for  tasking 
at  the  time  of  the  request,  (2)  they  have  a  high  probability  of  causing  significant  collateral 
damage,  and  (3)  it  takes  time  to  pass  the  target  information  and  receive  clearance  to 
engage  the  target — an  unacceptable  delay  when  engaging  a  fleeting,  high  value  target. 

These  drawbacks  have  led  to  research  in  the  areas  of  man-portable  Micro  Aerial 
Vehicles  (MAVs)  which  can  be  deployed  by  members  on  location,  requiring  minimal 
deployment  time  and  minimal  outside  coordination.  The  thesis  described  herein  is 
therefore  the  progeny  of  the  Air  Force  Institute  of  Technology’s  (AFIT’s)  response  to  the 
need  of  such  a  system.  This  thesis  is  the  culmination  of  the  theoretical  work  of  several 
other  groups,  the  implementation  of  some  previous  ideas  as  well  as  design  and  integration 
of  new  methodologies  determined  as  best  suited  for  completion  of  the  task  of  creating  a 
miniature,  mobile  system  which  can  track  and  engage  a  moving  target. 
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1.2  Problem  Statement 

Current  man-portable  weapon  systems  require  the  operator  to  have  unobstructed 
Line  of  Sight  (LOS)  to  the  target  at  effective  ranges  less  than  1000  meters.  Maintaining 
an  unobstructed  LOS  in  an  urban  environment,  while  staying  behind  protective  cover,  is 
challenging  at  best.  These  current  weapon  systems  also  require  the  operator  to  partially 
expose  themselves,  both  giving  away  their  location  and  exposing  them  to  enemy  fire.  In 
addition,  these  weapons  are  essentially  large,  explosive  bullets  with  no  loiter  or  wait 
capability.  With  the  current  capability,  a  team  tasked  with  engaging  a  time  sensitive 
target  in  a  small  city  would  have  to  infiltrate  the  city  undetected  to  within  approximately 
200  meters  of  the  target.  Assuming  the  ground  team  stayed  covert  while  moving  to 
intercept  the  target,  detection  is  almost  assured  once  the  current  weapon  systems  are  used 
to  engage  the  target.  Given  their  distance  from  friendly  forces,  the  survivability  of  the 
engaging  team  at  this  point  would  be  very  low.  Further,  if  the  target  moves,  or  the  ground 
force  team  is  re-directed,  their  response  time  is  comparably  higher  than  other  systems  due 
to  the  team’s  need  to  stay  covert  while  navigating  through  an  urban  environment. 

The  desired  organic  capability  is  a  responsive,  man-portable,  self-propelled,  low 
signature,  expendable  delivery  system  with  loiter  capability,  and  a  NLOS  range  greater 
than  that  currently  provided  by  fielded  systems.  This  new  system  would  allow  the  user  to 
covertly  launch,  loiter,  track,  positively  identify,  and  engage  a  time-sensitive,  high  value 
target  from  a  safe  distance.  The  system  should  be  effective  in  urban  environments  as  well 
as  desert,  maritime,  and  temperate  environments. 

1.3  Thesis  Purpose 

This  thesis  concentrates  on  providing  reliable  communications  throughout  all 
phases  of  the  Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  and  engagement 
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mission  by  introducing  an  additional  MAV  which  will  act  as  a  relay  between  the  ground 
team  and  the  envisioned  delivery  system.  The  presence  of  this  relay  MAV  should  not 
increase  the  operator  workload  and  should  therefore  fly  autonomously  to  maintain  the 
communication  link.  The  thesis  focuses  on  developing  the  guidance  laws  which  will 
dictate  the  behavior  of  the  relay  MAV. 

1.4  Thesis  Outline 

With  an  established  thesis  purpose,  the  remainder  of  the  thesis  will  now  focus  on 
describing  the  methodology  for  specifying  and  validating  the  developed  system.  The 
Background  chapter  provides  the  problem  background,  motivation,  and  an  overview  of 
related  research  work  in  the  fields  of  communication  relays  and  dynamic  games.  The 
Theoretical  Analysis  chapter  defines  the  full  dynamic  system  and  several  lower-order 
systems.  The  Demonstration  Hardware  and  Testing  chapter  describes  the  hardware 
components  and  flight  tests  associated  with  proving  the  theoretical  concept  of 
communication  relay.  The  Results  and  Analysis  chapter  verifies  that  the  developed 
system  meets  specified  requirements  and  validates  the  system  using  simulation  and  test 
results.  Finally,  the  Conclusions  and  Recommendations  chapter  discusses  lessons  learned 
and  recommends  areas  for  future  research. 
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2  Background 


2.1  Motivation 

Reliable  communication  is  essential  in  order  to  perform  the  ISR  and  engage 
mission.  The  envisioned  man-portable  system  will  not  be  supported  by  satellite 
communications,  but  will  use  radio  frequency  (RF)  modems.  High  frequency  radio 
communications  are  limited  by  an  adequate  Line-Of-Sight  (LOS)  between  the  operator  at 
the  Base  and  the  MAV.  The  MAV  systems  considered  in  this  thesis  are  utilized  to  seek 
out  and  engage  high  value  targets  and  will  be  referred  to  as  Rovers. 

There  are  a  number  of  environments  (e.g.,  urban,  forest,  mountainous)  in  which 
the  Base  may  often  lose  communication  with  deployed  Rovers  because  it  is  not  possible 
to  maintain  LOS  with  the  Rovers.  This  thesis  focuses  on  developing  guidance  laws  to 
optimally  position  a  specialized  Relay  MAV  to  provide  the  operator  at  the  Base  with  real¬ 
time  information  by  relaying  communication  and  sensor  data  when  there  is  no  LOS 
between  the  operator  and  the  Rover. 

2.2  Related  Research 

Recent  studies  have  produced  two  general  designs  for  a  reliable  and  robust 
communication  network  utilizing  mobile  communication  nodes.  The  first  design  consists 
of  one  or  more  mobile  communication  nodes  which  form  a  single  chain  to  relay 
information  between  the  source  and  the  destination.  This  is  referred  to  as  a  “single-flow 
network.”  Many  designs  for  single-flow  networks  use  a  fixed  source  and  fixed 
destination,  though  they  do  not  discount  the  possibility  of  a  mobile  destination  (Dixon 
and  Frew,  2007;  Goldenberg  et  al.,  2004).  The  second  design  consists  of  multiple  mobile 
communication  nodes  which  form  a  “mesh-like  network”.  This  configuration  adds  fault- 
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tolerance  for  a  more  robust  network  (Basu  and  Redi,  2004;  Floreano  et  al.,  2007). 
However,  a  “mesh-like  network”  would  be  ill  suited  for  the  envisioned  ISR  and 
engagement  system  due  to  desired  unit  covertness  while  engaging  a  high  value  target.  In 
this  respect,  Brown  et  al.  have  developed  the  Ad-hoc  UAV  Ground  Network  (AUGNet) 
test  bed,  showing  the  practicality  of  UAV-based  mobile  communication  nodes  using 
IEEE  802. 1  lb  wireless  routers  (Brown  et  al.,  2004).  The  proposed  ISR  and  engagement 
system  may  have  a  network  design  similar  to  AUGNet  but  the  Relay  must  still  have  an 
optimal  mobility  control  law  in  order  optimize  network  communications. 

Dixon  and  Frew  have  utilized  the  AUGNet  system  with  an  extremum  seeking 
controller  to  study  cooperative  electronic  chaining  while  maximizing  the  signal-to-noise 
ratio  between  the  nodes  of  the  multi-hop  network  (Dixon  and  Frew,  2007).  Goldenberg  et 
al.  have  shown  that  communication  nodes  should  be  evenly  spaced  on  the  line  between 
the  source  and  destination  in  order  to  minimize  the  energy  cost  of  communicating 
between  the  two  (Goldenberg  et  al.,  2004). 

2.3  Research  Statement 

The  main  distinction  of  this  work  is  that  no  cooperation  between  the  Rover  and 
the  Relay  is  imposed  while  optimally  positioning  the  communication  node  (the  Relay). 
This  scenario  is  then  modeled  by  posing  a  differential  game:  The  Relay  strives  to  position 
itself  such  that  the  RF  power  required  for  maintaining  communications  is  minimized, 
whereas  the  Rover  strives  to  position  itself  such  that  the  RF  power  is  maximized. 
Suboptimal  Relay  and  Rover  strategies  are  provided.  These  will  serve  as  a  first  guess  in 
solving  the  Two-Point  Boundary  Value  Problem  (TPBVP)  given  by  the  Pontryagin 
Maximum  Principle  and  which  yields  the  optimal  strategies. 
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3  Theoretical  Analysis 


3. 1  System  Definition 

It  is  assumed  that  the  rElay  (E)  MAV  is  cognizant  of  the  rOver’s  (O) 
instantaneous  position  relative  to  the  Base  (B)  as  well  as  its  own  position.  As  far  as  the 
RF  power  requirements  are  concerned,  this  is  determined  by  their  distance  from  the  Base 
and  the  Rover-Relay  separation.  Thus,  the  state  is  the  distance  rs  of  the  Relay  from  the 
Base,  the  distance  ro  of  the  Rover  to  the  Base,  and  the  angle  6  included  between  the 
radials  from  the  Base  to  the  Relay  and  the  Rover.  This  angle  is  measured  clockwise  (see 
Figure  1).  The  MAVs  have  simple  motion.  The  control  for  each  MAV  is  its  relative 
heading  angle  measured  clock-wise  from  its  radial  from  the  Base.  Figure  1  provides  a 
visualization  of  the  kinematics.  The  differential  equations  of  motion  are 

4  =  VE  cos  (p  ,r£(0)  =  r£o 

r0  =  V0cosiff  ,  r0(0)  =  r0o  >  (1) 

6  =  -^V0svny -^VEsm (p  ,  0(0)  =  0O,  0  <t<T 

T  is  the  planning  horizon  utilized  by  the  control  algorithm.  The  cost  functional  is 
indicative  of  the  RF  power  required  and  is  the  time  averaged  sum  of  the  squares  of  the 
distance  between  the  Relay  and  the  Rover  and  between  the  Relay  and  the  Base: 

\)  =  \TJ^Ed\t)  +  ^E\t)^dt 
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Figure  1 :  Schematic  of  Relay  System 


The  points  E,  B  and  O  in  K2  represent  the  positions  of  the  Relay,  Base  and 
Rover,  respectively.  These  three  points  form  a  triangle  which  can  be  utilized  to  calculate 

the  distance  EO  by  the  law  of  cosines. 


EO(t)  =  +rrj 


2 rEr0  cos  6 


Hence  the  cost  functional  is 


(2) 


The  relay’s  objective  is  to  minimize  the  average  RF  power  required  for 
maintaining  communications.  The  control  available  to  accomplish  this  task  is  limited  to 
setting  the  course  angle  (p  of  the  Relay,  while  the  Rover  does  whatever  it  wants,  namely, 
it  performs  the  ISR  mission:  in  a  worst  case  scenario,  one  might  assume  that  the  Rover  is 
working  to  maximize  the  cost  functional.  The  optimization  problem  is  then  a  differential 
game  where  the  Relay’s  control  is  its  relative  heading  (p  and  the  Rover’s  control  is  its 
relative  heading  y/. 

The  system  is  analyzed  by  first  non-dimensionalizing  the  states  and  the 
parameters.  The  velocities  are  scaled  by  the  velocity  of  the  Relay  (Ve),  yielding  a  non- 
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dimensional  speed  ratio  a  (a  =  jr)-  The  distances  are  scaled  by  the  initial  distance  of  the 


Rover  from  the  Base  so  that  each  distance  throughout  the  time  history  of  this  ISR  mission 
is  measured  in  units  of  .  Using  these  non-dimensional  parameters,  the  differential 

game  in  M2  x  S1  now  becomes 


min  max  Xf  =  £  ( 2r y  +  r(2  -  2 rErQ  cos  0}dt 


s.t. 

rE=cos<p  ,rE(0)  =  rEo 

r0  =  a  cosy/  ,  rQ( 0)  =  1 

9  =  -jrccsmy/ ,  9(0)  =  90,  0  <t<T 


(3) 


where  the  problem  parameter  is  the  speed  ratio  a  >  0 .  To  solve  the  differential  game,  the 
Hamiltonian  is  introduced  in  Equation  (4) 

9C  =  -2 rE  -r-Q  +  2 rEr0  cos  9  +  Xr<  cos  cp  +  Xt.a  cos  y/  +  Aff  (-^  a  sin  y/  -  sin  cp  j  (4) 

where  ,  Xr  and  X0  are  the  system  co-states. 

According  to  the  Pontryagin  Maximum  Principle  (Pontryagin  et  ah,  1962),  the 
differential  equations  for  the  co-states  are 


4  =  XrE  -  2r()  cos  9  -  ,  A^(T)  =  0 

rE 

l  =2ro-2Vos0+X‘aS'PV ,  l(T)  =  0 


Xe  =  2 rEr0  sin  9 


,  4,(70  =  0 


(5) 


and  the  optimality  condition  is  given  by  max  min  3C  ,  namely 

<p  v 


<99C  ,  .  X Q  cos  cp  . 

- =  -Xr  sin  cp — - - —  =  0 


dcp 


rE 


tan  cp*  =  — 


Xn 


(6) 


KrE 
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(7) 


S9C  ,  .  Am  cos  yr 

- =  -A,.  oc  sin  y/  +  — - 

dy/  0  rQ 

,  *  Ae 

=^>  tan  y/*  =  — — 


The  second-order  sufficiency  condition  for  cp  is 


<923C  ,  sin® 

~TT  =  ~K  c°s^  +  — - -<0 


and  inserting  the  expression  for  cp*  from  (6)  yields 


Ag(t)  <0  V  0  <t  <T 


Similarly, 


d2%  .  A„a  sin  yr 

—  /  /v  r*r\c'  l / /  ”  ' 


—  =  -A.  a  cos  «/■  - : 

,  A  'b  ' 


and  inserting  the  expression  for  yj*  from  (7)  yields 


MJ2 


=^>  Ae(t)  <  0  V  0  <t  <T 

The  expressions  for  cp *  and  y/*  given  in  Equations  (6)  and  (7)  can  also  be  used  to 
rewrite  the  state  and  co-state  equations  only  in  terms  of  the  states  and  co-states.  A 
standard,  albeit  nonlinear,  Two-Point  Boundary  Value  Problem  (TPBVP)  is  obtained  on 


the  interval  t  =  [0,  T\: 
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r, 


E 


r, 


e  = 


o 


r0  (0)  =  1 


(10) 


X  =  4 rE  -  2 rQ  cos  6  + 


Xb  =2 r0  -2 rE  cos  6 


Xg  =  2 rEr0  sin  6 


,  Xg(T)  =  0,  0 <t<T 


3.2  Suboptima!  Approaches 

Suboptimal  solutions  are  useful  in  their  own  right  and  provide  insight  into  the 
differential  game.  Suboptimal  solutions  can  also  be  used  to  provide  the  first  guess  for 
solving  the  TPBVP  (10)  of  the  optimal  control/differential  game. 

3.2.1  Geometric  Approach 

Using  a  geometric  approach  provides  a  suboptimal  but  easily  implementable 
solution  of  the  differential  game.  This  approach  is  suboptimal  because  the  Relay  and  the 
Rover  each  momentarily  assume  that  the  other  player  is  stationary  when  determining 
their  optimal  control. 


y 


Figure  2:  Schematic  of  Relay  System  Showing  the  Midpoint 
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The  geometry  of  the  engagement  forms  a  triangle  with  vertices  E,  B  and  O 
representing  the  respective  locations  of  the  Relay,  Base  and  Rover  (see  Figure  2).  Let  M 
be  the  midpoint  between  the  Rover  and  the  Base.  Simply  rotating  the  schematic  in  Figure 
2  provides  an  equivalent  schematic  (see  Figure  3)  which  is  similar  to  the  one  analyzed  in 
Appendix  A. 


Figure  3:  Schematic  of  Relay  System  Showing  Isocost  Circle 

If  the  Rover  were  stationary,  the  loci  of  constant  instantaneous  costs 

y  =  EO  +  BE 2 

for  the  Relay  are  concentric  circles  centered  at  the  midpoint  and  the  midpoint  is  the  Relay 
location  which  minimizes  the  cost  (Gutenmacher  and  Vasilyev,  2004).  The  Relay  is  on 
the  circumference  of  said  circles,  and  the  instantaneous  cost  y  is  detennined  by  the 

position  of  the  Relay.  This  means  that  the  gradient  vector  for  minimizing  cost  is  in  the 
radial  direction.  Therefore,  the  optimal  strategy  of  the  Relay  is  to  head  toward  the 
midpoint  M. 

The  optimal  control  of  the  Relay  is  determined  using  the  triangle  ABEM.  The 
distance  between  E  and  M  is  determined  using  the  law  of  cosines  (just  as  in  detennining 
the  distance  between  E  and  O  before).  The  control  angle  (p  is  then  found  indirectly  by 
finding  its  supplementary  angle  using  the  law  of  sines.  However,  due  to  an  inherent 
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ambiguity  in  the  law  of  sines,  the  control  law  is  specified  for  three  cases:  (1)  cp  is  acute, 
(2)  <p  is  90°  and  (3)  cp  is  obtuse. 


(p*  =  < 


sin 

n 

~2 

n  -  sin- 


r0  sin  6 


.[4r2 

V  £ 


V  "V  _r'£  ro  ^ rEro  cos  @ 


~  rn  cos  6 

for  rF  <  — - 

£  2 

rn  cos  6 


for  rE  =  0 


f 


rQ  sin  0 


yj4rE  -  4 rern  cos  6 


for  rE>  ° 


2 

rn  cos  6 


(ID 


This  ambiguity  can  be  bypassed  by  using  an  inverse  cosine  function  in  place  of 
the  inverse  sine.  Thus,  the  Relay’s  strategy  is: 

f  \ 


cp*  =  cos  1 


r0  cos  6  -  2 rE 


J 


y^4rl  +  r~  - 4rEr0  cos 9 
As  far  as  the  Rover  is  concerned  for  a  worst-case  scenario:  The  Rover  is  striving 


(12) 


to  maximize  the  cost  y  at  each  time  instant,  assuming  that  the  Relay  is  stationary.  This  is 


accomplished  by  expanding  the  isocost  circle.  The  most  effective  way  to  expand  the 
isocost  circle  is  to  increase  the  radius  EM  of  the  circle.  When  the  Relay  is  stationary,  the 
Rover  exclusively  controls  the  position  of  the  midpoint  M.  The  velocity  of  the  midpoint 
Mis  always  Vi  Vo,  where  Vo  is  the  velocity  of  the  Rover,  and  is  always  aligned  parallel  to 
the  velocity  vector  of  the  Rover  because  M  is  detennined  by  O,  and  their  motion  forms 
two  similar  triangles  (see  Figure  4). 


Figure  4:  Motion  of  M and  OForms  Similar  Triangles 
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The  Rover  would  like  to  head  in  such  a  direction  as  to  cause  the  midpoint  M  to 
move  such  that  the  radius  of  the  isocost  circle  increases  as  fast  as  possible.  Hence,  to 
achieve  this  result,  the  Rover,  which  controls  the  direction  of  the  velocity  vector  Vo, 
aligns  this  vector  with  the  line  EM ,  that  is,  it  heads  along  a  path  parallel  to  the  Relay. 
Thus,  the  Rover’s  strategy  is: 

y/*  =  (p*-Q  (13) 

Note  that  these  Relay  and  Rover  strategies  are  independent  of  the  planning 

horizon  T. 

3.2.2  One-Sided  Optimization 

The  complexity  of  the  dynamic  optimization  problem  is  significantly  reduced  by 
holding  one  of  the  MAVs  at  a  fixed  position:  We  will  optimize  the  control  of  the  Relay 
when  the  Rover  is  stationary,  that  is,  r0=  1 .  The  dynamics  are  the  same  as  previously 
developed,  but  now  the  parameter  a  =  0  and  the  state  space  is  reduced  to  M1  x  S1 .  The 
optimal  control  problem  is  considered: 

min  y  =  £  (lrE  +1-2 rE  cos  O^dt 

S'L  >  (14) 

=  cos  (p  ,rE(0)  =  rEo 

0  =  --^sin^>  ,  #(0)  =  0Q,  0  <  t  <  T  - 
The  new  Hamiltonian  is 

3C  =  -2 rE  -1  +  2 rE  cos  6  +  Xr<  cos (p - Xg  -^sin cp  (15) 

According  to  the  Pontryagin  Maximum  Principle  (Pontryagin  et  al.,  1962),  the 
differential  equations  for  the  co-states  are 
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Xr  =4rE-2cos6-AeSm(p  ,Xr(T)  =  0 


Xe  =  2 rE  sin  0 


UT)  =  0 


and  the  optimality  condition  is  given  by  max  3C ,  namely 


59C  ,  .  X9  cos  cp 

- =  -X„  sin® — - - 

d(p  E  rE 

X g 

=>  tan  cp  = - — 


The  second-order  sufficiency  condition  is: 


d2%  XnSmcp  A 

-^  =  -4  COS  (p  H  - - ^<0 


The  expression  for  cp  given  in  Equation  (17)  is  used  to  rewrite  the  state  and  co¬ 
state  equations  only  in  terms  of  the  states  and  co-states,  obtaining  the  nonlinear  TPBVP 


/;.(())  =  rK 


rEJKrE+^ 


,  m=e0 


X,  =  4r,  -  2  cos  6  +  - 


4K4+XI  rE 


,  X(T)  =  0 


Xe  =  2 rE  sin  9 


Xg(T)  =  0,  0  <t<T 


This  one-sided  optimization  problem  is  easier  to  solve  than  the  min-max  problem 
initially  posed  and  can  therefore  be  analyzed  using  typical  optimization  programs. 
GPOCS  is  a  Matlab-based  optimization  program  that  uses  the  “Gauss  pseudospectral 
method  where  orthogonal  collocation  is  performed  at  the  Legendre-Gauss  points” 
(Tomlab  Optimization  Inc.,  2008)  to  find  the  minimizing  path  of  the  Relay  in  this 


situation. 
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Appendix  B  also  provides  an  analysis  of  the  case  where  the  Rover  is  stationary, 
but  only  finds  the  optimal  location  of  the  Relay,  not  the  controls  which  will  guide  the 
Relay  to  that  location  (a  static  optimization  approach). 

An  additional  one-sided  optimization  problem,  not  considered  herein,  is  obtained 
when  the  Rover’s  point  of  view  is  taken,  namely,  the  Relay  is  stationary  and  the  Rover 
works  to  maximize  the  cost  functional. 

y = [(2rE  +  ro  -  2rEro cos  °) dt 

3.2.3  Suboptimal  Solution  Applied  to  TPBVP 

If  the  geometric  approach  proves  to  be  an  accurate  heuristic  method,  then  the 

initial  controls  that  coordinate  with  a  given  initial  state  can  be  found  using  the  results 
from  the  geometric  approach.  These  initial  controls  are  used  with  Equations  (6)  and  (7)  to 
find  a  relationship  between  the  initial  co-states.  This  relationship  reduces  the  number  of 
unknown  initial  co-states  to  one,  namely,  Ag. 

Choosing  an  appropriate  value  for  Ag  will  provide  the  two  other  initial  co-state 
values  needed  to  solve  the  nonlinear  TPBVP  given  in  Equation  (10).  Using  the  values  as 
initial  guesses,  the  optimal  initial  values  of  the  co-states  are  found  using  an  iterative 
method,  referred  to  as  a  “shooting”  method.  The  process  for  the  shooting  method  is  given 
below. 


1 .  At  the  iteration  step  k,  the  proposed  initial  co-states  (T(0)  =  /fi(0))  are  used  with  the 
known  initial  states  (x(0)  =  xfiO))  to  obtain  a  time  history  of  the  nonlinear  differential 
system  (10)  on  the  interval  0  <t<T  using  ode45  in  Matlab.  This  time  history  will  be 
referred  to  as  Xk(t)  and  AAt).  The  final  value  of  the  co-states  in  the  history  is  Ak(T). 

2.  The  differential  system  is  then  linearized  about  the  trajectory  xt(f),  Ak(t),  0  <  t  <  T  to 
obtain  a  time-dependent  linear  system  in  the  perturbations  Sxk(t),  8Ak(t): 


d_ 

dt 


f  Sx^ 

k8Xj 


=  4(0 


f  Sx^ 

k8Xj 


8x(0)  =  0,  8A(0)  =  8Ak,  0 <t<T 


(20) 


15 


The  Ak  matrix  is  found  using  the  Jacobian  function  on  the  differential  equations  of  the 
system  with  reference  to  the  states  and  co-states.  Then  the  values  of  the  states  and  co¬ 
states  are  substituted  into  the  At  matrix,  resulting  in  a  unique  Ak  matrix  at  each  time 
step. 

3.  Then,  each  of  the  states  and  co-states  are  give  an  initial  unit  perturbation  which  is 
then  propagated  using  the  linear  differential  system  to  find  the  resultant  change  in 
states  and  co-states  at  t  =  T.  This  collection  of  resultant  changes  is  combined  to  form 
a  resolvent  Q  matrix  which  relates  an  initial  perturbation  to  a  resultant  change  in 
state. 


'8x(J? 

,dX(T)y 


=  0(7) 


'  0  " 


(21) 


The  O matrix  can  actually  be  divided  into  four  sub-matrices: 

O =  ["°u  °u 

®2,1  °2,2 

^SA(T)  =  022(T)SAk  (22) 

4.  The  optimality  problem  (10)  requires  that  all  co-states  have  a  final  value  equal  to 
zero.  Therefore,  the  goal  is  for  any  nonzero  final  co-states  found  in  step  1  (A k(T))  to 
be  countered  by  the  resultant  change  in  co-state  found  in  step  3  ( 8A{T)). 

0  =  Ak(T)  +  8A(T) 

=  Ak  (T)  +  02  2  {T)SAk 

^SAk=-02x2(T)Ak(T)  (23) 

Then,  by  adding  the  perturbation  found  in  (23),  the  proposed  co-states  for  the  next 
iteration  should  result  in  a  final  co-state  value  of  zero. 

A+i  • =  \  + 

^Ak+l=Ak-^2{T)Ak{T)  (24) 


The  steps  in  this  shooting  method  are  then  repeated  until  the  final  co-states  A/c( T) 


converge  to  zero.  It  should  also  be  noted  that  if  the  sub-matrix  C^,2(l)  is  not  invertible, 


then  the  generalized  inverse  of  c^,2(l)  should  be  used: 

Specifically,  calculate  the  full  rank  factorization  of  02,2(1). 


022{T)  =  HK 

=»  ®\  2(T)  =  KT{KKTy\HTH)~xHT 
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The  shooting  method  yields  initial  co-states  which  do  not  equal  zero  at  the  end  of 
the  planning  horizon,  T.  Therefore,  the  co-states  are  used  as  initial  guesses  for  another 
shooting  method  program.  This  program  uses  ode45  to  solve  the  system  of  equations  (10) 
for  the  same  time  interval  (0  <  t  <  T)  and  iteratively  guesses  initial  conditions  for  the  co¬ 
states,  using  Isqnonlin,  to  minimize  the  error  of  the  terminal  conditions  of  the  co-states, 
namely,  the  co-states  must  equal  zero  at  T. 

This  final  shooting  method  provides  the  initial  co-states  which  most  closely  result 
in  satisfaction  of  the  terminal  constraints.  However,  the  performance  of  the  program 
requires  a  very  good  initial  guess.  The  first  shooting  method  provides  this  initial  guess. 
The  flow  chart  shown  in  Figure  5  provides  a  visualization  to  assist  in  understanding  how 
the  results  from  the  geometric  approach  are  used  by  the  aforementioned  Matlab  programs 
to  attain  full  system  results.  The  flow  represented  here  is  implemented  by  a  single  Matlab 
program:  “geometry_applied_rdg.m”. 


3.3  Planning  Horizon 

The  Rover  uses  the  services  of  the  Relay  as  long  as  OE  <  OB  ,  for  if  OE  >  OB  , 
communication  with  the  Base  through  the  Relay  will  be  counterproductive.  In  the  case 
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where  OE  =  OB  ,  the  geometry  of  the  engagement  will  form  an  isosceles  triangle  and  the 


Rover  will  lie  on  the  orthogonal  bisector  of  BE  shown  by  the  dashed  line  in  Figure  6. 


Figure  6:  Schematic  of  Initial  Condition  Border  Line 

Therefore,  in  order  to  make  proper  use  of  the  Relay,  the  state  must  satisfy  the 
condition 

O<0  <  cos-1  (25) 

The  optimal  solution  of  the  system  will  make  sense  for  planning  horizons  T, 
provided  that  the  state  satisfies  Equation  (25)  V  0  <  t  <  T.  The  geometric  considerations- 
based  suboptimal  strategies  in  Figure  7  show  that  eventually,  the  orthogonal  bisector  of 
the  segment  BE  will  be  crossed  by  the  Rover  and  hence  there  must  be  a  maximum 
planning  horizon,  depending  on  the  initial  values  for  rE,  ro  and  0.  Once  the  Rover  reaches 
the  bisector  of  the  segment  BE ,  the  game  is  over. 


Figure  7:  Initial  Condition  with  Limited  Planning  Horizon 
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However,  if  the  initial  state  is  as  shown  in  Figure  8,  then  the  maximum  possible 
planning  horizon  may  theoretically  approach  infinity.  That  is,  condition  (25)  will  not 
limit  the  maximum  planning  horizon;  in  practice,  the  planning  horizon  may  still  be 
limited  by  other  factors.  For  this  to  be  the  case,  the  state  must  satisfy  the  condition 


^<cos'1(2j) 


(26) 


Therefore,  the  maximum  possible  planning  horizon  is  unbounded  by  the  relation 


between  OE  and  OB  when 


0 <6 <  cos  1 

and  is  bounded  when 

cos-1  (2-^)  <  6  <  cos-1 

The  maximum  planning  horizon  may  also  be  bounded  by  the  nature  of  the  cost 
functional:  The  instantaneous  cost 

y  =  EO  + BE 2 

will  only  depend  on  BE  when  using  the  suboptimal  geometric  considerations-based 
strategies  for  the  Relay  and  Rover  because  EO  will  remain  constant  for  a  =  1  and  does 
not  depend  on  T.  The  planning  horizon  used  will  allow  BE  to  decrease  as  the  Relay 

moves  along  the  line  EM  until  BE  _L  EM  at  the  point  Ec,  as  shown  in  Figure  9.  Once 
this  point  is  reached,  lengthening  the  planning  horizon  provides  no  benefit  for  the  Relay. 
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Figure  9:  Schematic  of  Relay  Location  Closest  to  Base 

This  strongly  indicates  that  a  conjugate  time  ( Tc )  might  exist,  which  would  induce 
an  upper  bound  on  the  planning  horizon.  As  a  first  approximation. 


T  =- 


EK 

F, 


(27) 


The  distance  EEc  is  found  by  first  considering  the  area  of  the  triangle  ABEM. 


s  =  \r0rE  sin# 


At  the  same  time, 


S  =  \BEC  -EM 


2  yjrE  4  rO  rErO  C0S  ®  '  EE c 

Combining  Equations  (28)  and  (29)  yields 


BE  = 


rErQ  sin  6 


1  ylrE+iro-rEro  cos# 


Finally,  EEc  is  found  using  the  Pythagoras  Theorem: 


(28) 


(29) 


(30) 


EE  = 


rZE-BEc 


ri  _  rErQ  sin"  0 


E  4rE  +  rh  -  4 rErQ  cos  6 


EE,  =  rE 


4rE  -  4 rErQ  cos  6  +  rE)  cos°  6 
4 rE  -  4 rErQ  cos  6  + 


(31) 


Therefore,  the  approximate  conjugate  time  is  found  by  inserting  Equation  (31) 
into  Equation  (27) 
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T  _  4 r°-  -  4 rEr0  cos  0  +  r20  cos2  0  2 

VE[  4rE  -  4rErQ  cos  d  +  r^ 

and  will  require  T  <  Tc. 


(32) 
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4  Demonstration  Hardware  and  Testing 


4.1  Demonstration  Configuration 

The  hardware  for  the  demonstration  was  provided  by  the  AFIT  Advanced 
Navigation  Technology  (ANT)  Center’s  laboratory.  The  hardware  presented  here  will  not 
be  used  for  final  production  of  the  ISR  and  engagement  system.  It  only  serves  to  test  and 
demonstrate  the  discussed  concepts. 

4.1.1  Unmanned  Aircraft 

One  of  the  standard  aircraft  flown  by  the  ANT  Lab  is  the  SIG  Manufacturing 
Company  Rascal  110.  This  aircraft  has  very  stable  flight  characteristics  and  a  large 
payload  capacity.  It  has  a  braced  high- wing  configuration  with  a  wingspan  of  1 10  inches 
and  a  tail  wheel.  The  initial  flight  testing  of  the  Rascal  for  the  ANT  Lab  was  perfonned 
by  Jodeh  in  2006.  The  resulting  performance,  stability  and  static  data  were  also  presented 
by  Jodeh  (Jodeh,  2006). 


Figure  10:  Sig  Rascal  110  (©2008  Tower  Hobbies) 

4.1.2  Autopilot 

The  autopilot  chosen  for  this  demonstration  is  the  Kestrel  Autopilot  System  v  2.2 
produced  by  Procerus  Technologies.  The  autopilot  is  very  compact  and  weighs  only  16.7 
grams,  making  it  “the  smallest  and  lightest  full  featured  autopilot  on  the  market” 
(Procerus,  2008).  This  autopilot  was  chosen  because  it  will  be  able  to  fit  in  any  aircraft 
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chosen  for  the  final  ISR  and  engagement  system.  It  was  also  chosen  because  Procerus 
Technologies  also  offers  high  quality  compatible  software  (OnPoint  Targeting)  for  video 
analysis  and  target  recognition.  The  autopilot  is  controlled  through  the  included  Virtual 
Cockpit  program.  This  program  easily  accepts  latitude  and  longitude  coordinates  to  set 
waypoints  for  direct  flight. 


Figure  1 1 :  Kestrel  Autopilot  System  (©2008  Procerus  Technologies) 

4.1.3  Communications  Transmission 

The  standard  radio  frequency  modem  used  by  Procerus  for  the  Kestrel  autopilot  is 

the  Xtend  OEF  RF  module  available  from  Digi  International  (formerly  MaxStream). 
These  modems  operate  in  the  900  MHz  frequency  band  and  have  an  output  power  of  one 
watt.  The  outdoor  line-of-sight  range  is  up  to  40  miles  (with  a  high  gain  antenna)  and  it 
supports  a  repeater  network  topology  (Digi  Int.,  2008). 
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Figure  12:  Xtend  OEF  RF  Module  (©2007  Digi  International) 


4.1.4  Sensor  Data  Transmission 

The  system  used  to  transmit  and  receive  data  from  the  on-board  camera  is  a 

product  of  Black  Widow  AV.  This  set  transmits  on  one  of  eight  channels  in  the  2.4  GHz 
frequency  band  with  an  output  power  of  one  watt.  Each  unit  in  the  set  can  operates  on  a 
12  V  battery  pack  while  the  receiver  also  comes  with  an  AC  adapter  (Black  Widow  AV, 
2008). 


Figure  13:  Black  Widow  1  W  Transmitter  and  Receiver  Set  (©2007  Black  Widow  AV) 
4.1.5  Relay  Configuration 

The  relay  for  the  sensor  data  was  the  only  relay  available  for  the  proof  of  concept 
demonstration.  The  hardware  and  software  required  to  test  a  relay  for  autopilot  and 
control  signals  was  unavailable.  The  relay  consisted  of  a  Black  Widow  transmitter  and 
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receiver  set,  where  the  output  of  the  receiver  was  directly  fed  into  the  input  for  the 
transmitter  (by  the  long  blue  cord  seen  in  Figure  14,  below). 


Figure  14:  Setup  for  Sensor  Data  Relay  Demonstration 


In  this  configuration,  the  transmitter  (seen  on  the  left  side  of  Figure  15) 
rebroadcast  the  data  from  the  receiver  (seen  on  the  right  side  of  Figure  15)  on  a  different 
channel.  The  receiver  and  transmitter  were  also  separated  so  as  to  avoid  any  interference 
between  them. 


Figure  15:  Detail  of  Setup  for  Sensor  Data  Relay  Demonstration 

4.1.6  Implementation  of  Relay  MAV  Guidance 

The  theoretical  analysis  showed  that  the  loci  of  constant  instantaneous  costs 

y  =  EO  +  BE 2 

for  the  Relay  are  concentric  circles  centered  at  the  midpoint  of  the  straight  line 
connecting  the  Base  to  the  Rover  and  that  this  midpoint  is  the  Relay  location  which 
minimizes  the  cost  (Gutenmacher  and  Vasilyev,  2004).  Furthermore,  the  gradient  vector 
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for  minimizing  the  cost  is  in  the  radial  direction.  Therefore,  the  simplest  and  most 
effective  way  to  minimize  the  instantaneous  cost  is  for  the  Relay  to  fly  straight  to  the 
midpoint. 

In  the  envisioned  ISR  and  engagement  system,  the  controls  for  both  the  Rover  and 
the  Relay  will  come  from  the  same  ground  control  unit  (GCU).  The  GCU  will  also 
process  and  record  the  sensor  data  from  the  Rover.  In  order  to  reduce  the  computational 
demand  on  the  GCU,  the  implementation  of  the  optimal  Relay  guidance  will  only  consist 
of  a  program  which  calculates  the  position  of  lowest  cost  at  a  particular  time  and  then 
provides  this  position  to  the  Virtual  Cockpit  program  as  a  loitering  waypoint.  This 
program  is  called  “commrelay.m”  (see  Appendix  C)  and  is  systematically  run  every  few 
seconds  or  every  time  the  signal  strength  drops  below  a  specified  threshold.  This  method 
does  not  truly  implement  the  optimal  guidance  laws  derived  in  the  theoretical  analysis, 
but  it  does  guide  the  Relay  to  fly  straight  to  the  optimal  location  at  a  particular  time 
without  inhibiting  the  performance  of  the  rest  of  the  system. 

4.2  Testing  Procedure 

The  relay  was  set  up  on  a  six-foot  long  folding  table  on  top  of  a  berm  near  the 
runway.  Both  the  aircraft  transmitter  and  the  relay  receiver  were  set  to  channel  one.  The 
relay  receiver  then  output  the  signal  to  the  relay  transmitter,  which  was  set  to  channel 
four.  The  operator  at  the  ground  station  verified  that  both  channels  showed  the  same  data 
by  switching  between  channel  one  and  channel  four.  For  this  test,  the  receiver  and 
transmitter  of  the  relay  were  six  feet  apart,  which  is  one  foot  shorter  than  the  overall 
length  of  the  SIG  Rascal  110.  The  test  was  repeated  with  the  receiver  and  transmitter  of 
the  relay  only  four  feet  apart  to  show  a  feasible  range  for  smaller  aircraft. 
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5  Results  and  Analysis 


5.1  Numerical  Results 

The  numerical  results  shown  below  were  all  found  using  Matlab. 


5.1.1  Suboptimal  Geometric  Results 

The  following  numerical  results  illustrate  the  evolution  of  the  differential  game  in 
the  case  where  T  =  0.25,  a  =  1,  =  0.5,  and  60  =  f .  In  the  game  plane,  the  Rover 

always  starts  from  (x,  v)  =  (1,  0).  The  trajectories  show  a  visualization  of  the  state  history, 
where  the  Base  location  is  designated  by  a  star  at  the  origin  and  the  final  location  of  each 
MAV  is  signified  by  a  triangle.  The  final  midpoint  location  is  designated  by  a  square. 

Reduced  State  Space:  State  History 
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Figure  16:  Geometric  Time  History  for  T  =  0.25,  a  =  1,  rEn  =  0.5,  and  6()  =  6 
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Trajectories  in  Game  Plane 
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Figure  17:  Geometric  Spatial  Results  for  T  =  0.25,  a  =  1,  =  0.5,  and  6*0  =  f 

The  following  numerical  results  illustrate  the  evolution  of  the  differential  game  in 
the  case  where  T  =  0.48,  a  =  1,  rEo  =  0.5,  and  60  =  f . 
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Reduced  State  Space:  State  History 
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Figure  18:  Geometric  Time  History  for  T  =  0.48,  a  =  1,  =  0.5,  and  0()  =f 
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Trajectories  in  Game  Plane 
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Figure  19:  Geometric  Spatial  Results  for  T  =  0.48,  a  =  1,  r£  =  0.5,  and  0O  =f 

The  results  from  the  scenarios  above  show  that  the  system  performed  exactly  as 
designed. 

If  T  is  sufficiently  large,  the  three  points  E,  B  and  O  might  become  collinear. 

Once  the  three  points  are  collinear  (0=0)  the  motion  is  confined  to  a  straight  line.  The 
Relay  moves  toward  the  midpoint  M  and  the  Rover  moves  away  from  the  Relay.  If  a  <  2 
the  Relay  will  need  to  slow  down  once  it  reaches  the  midpoint. 

The  following  numerical  results  illustrate  the  evolution  of  the  differential  game  in 
the  case  where  T  =  0.5,  a  =  2,  r£  =  0.5,  and  0O  =  f . 
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Reduced  State  Space:  State  History 
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Figure  20:  Geometric  Time  History  for  T  =  0.5,  a  =  2,  r£  =  0.5,  and  <90  =f 
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Figure  21 :  Geometric  Spatial  Results  for  T  =  0.5,  a  =  2,  /-£  =  0.5,  and  0o=f 
Once  E,  B  and  O  are  collinear,  the  reduced  Relay  velocity  eliminates  the  need  for 
excessive  control  use.  However,  it  is  possible  that  the  Relay  might  never  arrive  at  the 
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midpoint  due  to  a  short  planning  horizon  T,  or  the  maximizing  efforts  of  the  Rover.  If  the 
Rover  used  a  suboptimal  control  strategy  (which  is  common  in  practice),  it  is  possible  for 
the  Relay  to  arrive  at  the  midpoint  and  consistently  match  the  motion  of  the  midpoint. 

5.1.2  One-Sided  Optimization  Results 

The  following  numerical  results  show  the  solution  of  the  optimization  problem  for 
T  =  0.25,  rE  =  0.5,  and  0O  =  -f-.  The  co-state  initial  conditions  satisfying  the  terminal 

constraints  are  Ar/  (0)  =  0.0347  and  Ao(0)  =  0.0647. 


Reduced  State  Space:  State  History 
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Trajectories  in  Game  Plane 


Figure  23:  One-Sided  Spatial  Results  for  T  =  0.25,  =  0.5,  and  0()  =f 

It  is  important  to  note  that  the  slope  of  the  path  traveled  by  the  Relay  maintains  a 
constant  value  of  105°  (measured  counter-clockwise  from  the  x-axis).  The  slope  time 
history  of  the  path  has  a  numerical  standard  deviation  of  0.0153°.  This  shows  that  the 
Relay  travels  along  the  straight-line  path  toward  the  midpoint  M.  This  corroborates  nicely 
with  the  heuristic  solution  found  using  the  geometric  approach. 

The  following  numerical  results  show  the  solution  of  the  optimization  problem  for 
T  =  0.48,  rE  =  0.5,  and  0O  =  f.  The  co-state  initial  conditions  satisfying  the  terminal 

constraints  are  An  (0)  =  0.25  and  Ad(0)  =  0.2165. 
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Reduced  State  Space:  State  History 
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Figure  24:  One-Sided  Time  History  for  T  =  0.48,  rEo  =  0.5,  and  0o=f 
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Figure  25:  One-Sided  Spatial  Results  for  T  =  0.48,  rE  =  0.5,  and  90  = 
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Similar  to  the  first  case,  the  slope  of  the  path  traveled  by  the  Relay  maintains  a 
constant  value  of  120°  (measured  counter-clockwise  from  the  x-axis).  The  slope  time 
history  of  the  path  has  a  numerical  standard  deviation  of  0.0193°.  This  shows  that  the 
optimal  solution  is  indeed  a  straight  line  path  toward  the  midpoint  between  the  Base  and 
the  Rover. 

5.1.3  Results  Using  Complete  System 

The  suboptimal  solutions  provide  corroborating  results,  showing  that  the  optimal 
control  of  the  Relay  is  to  fly  directly  toward  the  midpoint  between  the  Rover  and  the 
Base.  The  optimal  control  for  the  Rover  to  reverse  this  action  would  be  to  fly  parallel  to 
the  Relay,  as  suggested  in  the  suboptimal  geometric  consideration.  Therefore,  the 
nonlinear  TPBVP  given  by  Equation  (10)  is  solved  using  the  two  shooting  methods 
described  previously.  These  shooting  methods  find  the  initial  co-states  by  using  the  data 
obtained  from  the  geometric  approach  with  Equations  (6)  and  (7),  and  an  appropriate 
value  for  Xg.  To  provide  an  easier  comparison,  the  game  plane  results  of  the  geometric 
approach  are  repeated  with  the  results  found  using  the  shooting  methods  in  the  figures 
below. 

The  following  numerical  results  show  the  solution  of  the  differential  game  where 
T  =  0.25,  a  =  l,  =  0.5  and  d0  =  f.  The  initial  co-states  which  most  closely  satisfy  the 

tenninal  constraints  are  Xr/  (0)  =  -0.0185,  /fy  (0)  =  -0.3 126  and  X&(0)  =  -0.0838.  The 
resulting  tenninal  co-states  are  -fy  (T)  =  0.0,  Xro  (T)  =  0.0018  and  X0(T)  =  -0.0001. 
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— 1 - 1 - 1 - 1 - r 

- Geometric-Relay 

\ 

- Shooting-Relay 

0.2 

u 

\\ 

- Geometric-Rover 

Shooting-Rover 

0.1 

I  \ 

Geometric-Midpoint 

\  \ 
l  \ 

Shooting-Midpoint 

l  ^ 

A  Base 

0 

A  ^ 

. . 

\  . 

0.1 

1 

1 

1 

1 

0.2 

- 

\ 

\ 

4 

0.3 

- 

_ 1 _ 1 _ 1 _ 1 _ L 

_ 1 _ 1 _ 

0  0,2  0.4  0.6  0.8  1  1.2 


x 

Figure  27:  Comparative  Spatial  Results  for  T  =  0.25,  a=  1,  =  0.5  and  <90  =  f 

These  results  differ  from  the  geometric  approach,  but  the  geometric  approach  is 
suboptimal.  This  may  be  the  closest  result  to  an  optimal  solution  to  the  differential  game 
for  the  given  initial  condition,  since  the  second-order  sufficiency  conditions  (given  by 
equations  (8)  and  (9))  are  satisfied  and  the  terminal  co-states  are  very  near  zero.  It 
appears  that  the  Relay  is  still  heading  toward  M,  but  the  Rover’s  strategy  has  changed 
considerably. 

The  results  in  Figures  28  and  29  show  the  solution  of  the  differential  game  where 
T  =  0.48,  a  =  1,  rE  =  0.5  and  O0  =  -f .  The  initial  co-states  which  most  closely  satisfy  the 

terminal  constraints  are  fy  (0)  =  -0.41 12,  /fy  (0)  =  -0.9082  and  2^(0)  =  -0.3875.  The 

resulting  terminal  co-states  are  /fy  (T)  =  -0.0,  2,  (7")  =  -0. 1 007  and  2^(7")  = -0.1018. 
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These  results  also  differ  from  the  geometric  approach.  It  appears  that  the  Relay  is 
no  longer  heading  toward  M,  but  the  terminal  co-states  also  have  a  greater  error  from  the 
constraints.  The  second-order  sufficiency  conditions  (given  by  equations  (8)  and  (9))  are 
still  satisfied. 


Reduced  State  Space:  State  History 
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Figure  28:  Full  System  Time  History  for  T  =  0.48,  a  =  1,  =  0.5  and  0o=f 
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Trajectories  in  Game  Plane 
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Figure  29:  Comparative  Spatial  Results  for  T  =  0.48,  a  =  1,  rEg  =  0.5  and  0o=f 

The  results  in  Figures  30  and  3 1  show  the  solution  of  the  differential  game  where 
T  =  0.5,  a  =  2,  rE  =  0.5,  and  0O  =  f .  The  initial  co-states  which  most  closely  satisfy  the 

terminal  constraints  are  Xrs  (0)  =  -15.3439,  Xro(0)  =  1.8354,  and  X0(O)  =  -0.0628 .  The 
resulting  terminal  co-states  are  Xr/  (T)  =  -0.0,  Xr(T)  =  -0.0141  and  A e(T)  =  0.0209. 

These  results  were  much  unexpected.  The  scenario  represents  a  special  case 
where  the  Relay  moves  at  the  same  speed  as  M  (a  =  2).  Therefore,  these  results  are 
possibly  due  to  unforeseen  physics  present  in  the  system.  The  results  may  also  be  due  to 
an  improper  value  for  the  planning  horizon,  T.  The  terminal  co-states  are  very  near  zero 
but  the  second-order  sufficiency  conditions  are  not  satisfied,  since  Xg  starts  out  negative 
and  ends  positive.  Therefore,  it  may  be  possible  that  this  result  gives  the  opposite  goal  of 
optimization,  namely,  the  Relay  ends  up  trying  to  maximize  the  cost  while  the  Rover  tries 
to  minimize  the  cost.  This  may  also  be  simply  due  to  the  fact  that  a  sixth  order,  nonlinear 
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TPBVP  is  very  complex  and  non- intuitive.  Regardless  of  these  conjectures,  no  observed 
initial  values  for  /t^havc  provided  results  which  differ  from  those  seen  here.  This 
provides  more  incentive  for  further  research  of  the  system  and  the  nonlinear  TPBVP 

given  by  Equation  (10). 

Reduced  State  Space:  State  History 
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Figure  30:  Full  System  Time  History  for  T  =  0.5,  a  =  2,  rE  =  0.5,  and  60  =  f 
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Trajectories  in  Game  Plane 
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Figure  31:  Comparative  Spatial  Results  for  T  =  0.5,  a  =  2,  r£  =  0.5,  and  60  =  f 
5.2  Test  Results 

The  concept  demonstration  for  the  sensor  data  relay  was  perfonned  at  Camp 
Atterbury  Army  National  Guard  Base  in  Indiana  on  November  9,  2007.  During  the  first 
test,  the  rebroadcast  signal  from  the  relay  had  some  static  but  showed  the  same  picture  as 
the  original  signal  being  broadcast  from  the  aircraft.  This  static  can  be  avoided  in  future 
applications  by  using  a  higher  quality  receiver  for  the  relay  (e.g.,  a  diversity  receiver). 

The  test  was  repeated  with  the  receiver  and  transmitter  of  the  relay  only  four  feet  apart. 
This  change  in  configuration  gave  the  same  results  as  the  first  test.  Therefore,  if  the  Relay 
has  at  least  four  feet  of  usable  space  in  the  fuselage  of  the  aircraft,  interference  should  not 
be  a  main  cause  of  concern. 

The  practical  Implementation  of  the  Relay  guidance  has  been  proven  using  a 
Hardware-in- the-Loop  simulation  with  the  Fleeting  Target  Controller  developed  by 
Sakryd  and  Ericson  in  conjunction  with  this  study. 
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6  Conclusions  and  Recommendations 

6.1  Conclusions 

This  thesis  developed  optimal  guidance  laws  for  a  Relay  MAV  in  support  of  ISR 
beyond  LOS.  The  guidance  laws  are  based  upon  the  solution  of  a  min-max  optimization 
problem,  namely,  the  solution  of  the  differential  game,  which  represents  a  worst  case 
scenario.  The  solution  of  the  differential  game  hinges  on  the  solution  of  a  nonlinear 
TPBVP. 

Suboptimal  Relay  (and  Rover)  guidance  strategies  are  first  provided.  The  first  of 
these  suboptimal  guidance  strategies  is  derived  using  a  geometry-based  (sub)optimality 
principle.  The  Relay  heads  toward  the  instantaneous  midpoint  of  the  straight  line  between 
the  Rover  and  the  Base.  The  Rover,  heads  away  from  the  Relay  on  a  course  parallel  to 
that  of  the  Relay.  The  second  suboptimal  guidance  strategy  is  a  one-sided  Relay  optimal 
control  problem,  where  the  Rover  is  considered  stationary.  The  results  using  the  Matlab 
optimization  program  GPOCS  showed  that  the  optimal  guidance  of  the  Relay  is  to  fly 
directly  toward  the  instantaneous  midpoint  of  the  straight  line  between  the  Rover  and  the 
Base. 

The  heuristic  methods  provided  corroborating  results  which  were  then  used  as 
first  guesses  for  a  combination  of  two  shooting  methods  in  series  to  solve  the  TPBVP. 

The  shooting  method  results  yielded  the  approximate  numerical  solution  of  the  Relay- 
Rover  differential  game. 

The  practical  Implementation  of  the  Relay  guidance  consists  of  a  Matlab  program 
which  calculates  the  Relay  position  which  requires  the  lowest  RF  power  (i.e.,  the 
midpoint  between  the  Rover  and  the  Base)  at  a  particular  time  and  then  provides  this 
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position  to  the  Virtual  Cockpit  program  as  a  loitering  waypoint.  The  program  relies  on 


the  Kestrel  autopilot  to  fly  toward  the  waypoint  and  loiter  there  until  it  receives  a  new 
loitering  waypoint.  This  program  has  been  proven  using  a  Hardware-in-the-Loop 
simulation  with  the  Fleeting  Target  Controller.  The  hardware  configuration  required  to 
relay  sensor  data  consisted  of  a  ground-based  receiver/transmitter  set  which  received  the 
data  and  fed  it  directly  to  the  transmitter,  which  transmit  it  on  a  different  channel.  This 
configuration  was  tested  and  proved  that  the  concept  is  feasible  for  future  use  on  an 
aircraft. 


6.2  Recommendations 

The  following  recommendations  would  help  provide  more  robust  and  realistic 
guidance  laws  for  the  system. 

•  Analyze  the  relationship  between  the  initial  states  and  initial  co-states  to  find  a 
more  routine  method  to  find  a  solution  to  the  TPBVP. 

•  Use  a  different  method  (i.e.,  different  from  the  “shooting  method”)  to  analyze 
the  complete  system. 

•  Perform  another  one-sided  optimization  from  the  Rover’s  point  of  view  (i.e., 
trying  to  maximize  the  cost  while  the  Relay  is  stationary. 

•  Use  a  more  realistic  system  model,  which  considers  wind  effects  and  aircraft 
attitude. 

•  Incorporate  more  realistic  constraints  on  the  differential  equations  of  motion 
(e.g.,  maximum  turn  rate,  stall  speed,  etc.). 

•  Consider  the  signal  strength  or  signal-to-noise  ratio  as  a  driving  factor  for 
finding  the  optimal  relay  location  and  guidance  laws. 

•  Analyze  the  effect  of  the  speed  ratio  a  and  determine  if  there  is  an  optimal 
value  for  both  arriving  at  the  optimal  location  and  matching  its  movements. 
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Appendix  A  -  Geometry 

An  Elementary  Euclidean  Geometry  Result: 

It  is  well  known  that  the  locus  of  all  points  such  that  the  sum  of  the  distances  from 
two  fixed  points  is  constant,  is  an  ellipse.  Thus,  the  following  is  of  some  interest. 

Theorem  1  The  Locus  of  all  points  such  that  the  sum  of  the  squares  of  the 
distances  from  two  fixed  points  is  constant,  is  a  circle  centered  at  the  midpoint  of  the 
segment  formed  by  the  two  fixed  points.  The  radius  of  this  circle  is 

R  =  jd2-f2 
2 

where  the  sum  of  the  squares  of  the  distances  is  2d '  and  the  distance  between  the  fixed 
points  is  2/;  obviously,  d>f. 

Proof: 

Let  the  fixed  points  F\  and  F2  be  on  the  x-axis  (F\  =  if,  0),  F2  =  {-f  0))  as  shown 
in  the  figure  below. 


y 


Figure  32:  Schematic  of  Fixed  Points  Showing  Isocost  Circle 

The  sum  of  the  squares  of  the  distances  is  calculated  as 
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2d2  =(/  +  x)2  +  v2  +  (/  -  x)2  +  v2 
=  2f2  +  2x2  +  2  v2 

2,2  >2  z-2 

=>  x  +y  =d  -f 

This  is  the  equation  of  a  circle  centered  at  the  origin,  whose  radius  is 

R  =  \jd2  -  f2 

□ 

This  result  appeared  in  Gutenmacher  and  Vasilyev  (2004). 

Remark:  The  loci  of  constant  costs,  2d  ,  are  concentric  circles  where  the 
minimum  cost  is  found  at  the  midpoint  of  the  line  formed  by  F\  and  F2,  where  d  =f 

Extension:  The  Locus  of  all  points  such  that  the  weighted  sum  of  the  squares  of 
the  distances  from  two  fixed  points  is  constant,  is  a  circle  centered  on  the  segment  fonned 
by  the  two  fixed  points  and  is  at  a  distance  of  (1  -  2 a)/ from  this  segment’s  midpoint. 

The  radius  of  this  circle  is 

R  =  ^d2  -4a(\-a)f2 

where  d2  is  the  specified  weighted  sum  of  the  squares  of  the  distances,  the 
distance  between  the  fixed  points  is  2/;  and  the  weight  is  a;  if  a  <  0  or  a>  1  this  is  true 
\/d  >  0 ,  and  if  0  <  a  <  1 ,  d  >2f  ^a(l-a) .  Note:  When  the  weight  a  =  lA,  need  d  >  f . 

Proof:  The  weighted  sum  of  the  squares  of  the  distances  is  calculated  as 

d1  =a[(/  +  x)2+/]  +  (l-a)[(/-x)2+y2] 

=  af2+  ax2  +2 afx  +  ay2  +  (1  -  a)f2  +  (1  -  a)x2  -  2(1  -  a)fx  +  (1  -  a)y2 
=  f2+x2+  y2  - 2fx(l  - 2a) 

=  [x  -  (1  -  2a)  ff  +f2+  y2  -  (1  -  2a)2  f2 
=>  [x  -  (1  -  2a) /]'  +  y2  =  d2  -  4a(l  -  a)  f 2 

□ 
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Appendix  B  -  Static  Optimization  Approach 

Observing  the  system  statically  can  provide  insight  on  the  differential  game  by 
providing  the  optimal  placement  of  the  MAVs  if  their  motion  were  momentarily  frozen. 
This  approach  will  provide  the  optimal  placement  of  the  Relay  relative  to  the  Rover 
location.  The  goal  is  to  minimize  the  cost  functional  given  in  Equation  (1)  below,  which 
is  the  integrand  of  the  original  differential  game.  Therefore,  the  static  optimization 
problem  is 


min  y (rE ,  8)  =  2rE  +r^~  2 rEr0  cos  8  \ 
s.t. 

gl(rE,0)  =  -rE<O 
g2{rE,0)  =  -n-0<  0 
gi(rE,0)  =  0-7r<O 

Transforming  the  problem  into  a  Lagrange  fonn  gives 


(1) 


L(rE,8,A,s )  =  2 rE  +  r°  -2 rErQ  cos8  +  vl{-rE  +  sx)  +  u2(-7r  -  8  +  s^)  +  u3(8  -  7t  +  s3)  (2) 
The  first-order  KKT  necessary  conditions  for  optimality  (Karush,  1939;  Kuhn  and 

Tucker,  1951)  for  this  function  are 


L  (rE ,  8,  v,  s)  =  0  =>  4 rE  -  2 r0  cos  0  -  vx  =  0 
Le(rE,6,v,s )  =  0  2  rEr0  sin  0-u2+u3  =  0 


-rE  +  s\ 

"o' 

(Te  ">  *^)  ^ 

—71  —  8  + 

= 

0 

8 -71 +  s\ 

0 

2sxvx 

"0" 

Ls(rE,0,v,s)  =  0=> 

2  s2o2 

= 

0 

2s3u3 

0 

J 

The  set  of  equations  given  in  (3)  can  be  solved  because  there  are  eight  unknowns 
in  eight  equations.  However,  the  problem  will  first  be  solved  without  constraints  to 
determine  if  the  enforcement  of  said  constraints  is  necessary.  The  first  two  equations  in 
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(3)  are  solved  using  fsolve  in  Matlab  with  the  Lagrange  multipliers  set  to  zero  ( u,  =  0). 
Matlab  provides  the  following  output: 

[  rEopt,  thetaopt,  cost] 

[  l/2*rO,  0,  1 / 2 *rOA2 ] 

[  0,  -pi/2,  rOA2] 

[  0,  pi/2,  rOA2] 

Each  row  of  the  output  represents  a  stationary  point  of  the  system  which  could 
provide  a  maximum  or  minimum  cost.  None  of  these  stationary  points  violate  the  given 
constraints  so  solving  the  full  set  of  eight  equations  is  unnecessary.  The  first  of  the  three 
stationary  points  is  the  minimum  (as  shown  by  the  cost  given  in  the  last  column). 
Therefore,  given  the  Rover’s  position,  the  midpoint  between  the  Rover  and  the  Base  is 
the  optimal  location  of  the  Relay  to  minimize  the  integrand  for  the  differential  game.  This 
result  corroborates  with  the  work  done  by  Goldenberg  et  al.  (2004). 
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Appendix  C  -  Practical  Relay  Guidance  Code 

Commrelay  Program 

function  [wpx,  wpy] =commrelay (uavx, uavy) 

%  Communication  Relay  MAV  Positioning 

%  This  function  returns  the  x  &  y  coordinates  of  the  optimal  location  for  a 
%  Relay  MAV  given  the  x&y  coordinates  (relative  to  the  Base)  of  the  Rover 
%  MAVs  flying  the  ISR  mission.  The  x&y  coordinates  returned  are  best  input 
%  as  a  waypoint  for  the  autopilot  to  fly  toward/loiter  around. 

%  by  John  Hansen,  05  November  2007 

%  modified:  10  Dec  07,  JH :  added  robustness  for  inputs 


if  nargin==2 

[r,  c]  =size (uavx) ; 
if  r<c 

uavx=uavx 1 ; 
uavy=uavy 1 ; 

end 

P= [uavx,  uavy] 
elseif  nargin==0 

disp ( 1  WARNING :  No  inputs  received!  Proceeding  using  arbitrary 
location  1 )  ; 

disp ( 1  for  two  MAVs .  '  )  ; 

P=  [20  40; -20  40] 

else 

disp ( 1  Incorrect  number  of  inputs.  Need  two  inputs:  '); 
disp('l:  x-values  of  MAV(s)'); 
disp('2:  y-values  of  MAV(s)'); 

end 

%%  Initial  Guess 

x0= [mean (P(:,l)/2)+l /mean (P(:,2)/2)-l]  ; 

%%  Optimization 

options  =  optimset (' display ',  1  iter GradObj ',  1  of f 1 GradConstr ',  1  of f 1 ,  ... 

1 DerivativeCheck 1 , ' of f ' , 1 TolCon ' ,  le-8,  'TolX1,  le-8); 
[x, mincost , Exit Flag, Output] =fminunc (@ob  j ective , xO , options , P)  ; 

Output 
wpx=x ( 1 ) 
wpy=x (2 ) 
return 

%%  the  cost  function%%%%%%%%%%%%% 
function  [cost] =obj ective (x, P) 
n= length (P(: ,1) ) ; 
f 0=n*x (1) A2+n*x (2) A2 ; 
for  j=l:n 

f ( j )  =  (X (1)  -P ( j , 1)  ) a2+ (x (2)  - P ( j , 2 )  ) A 2  ; 

end 

cost=f 0+sum (f ) ; 
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